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Editing Video Data 

Background of the invention 

1 . Field of the Invention 

The present invention relates to editing original video data of a first 
fomiat, wherein said first format has inserted jitter frames and is displayable 
at a first frame rate. 

2. Description of the Related Art 

It is well known ttiat commercial video systems operate at thirty frames 
per second, slightly less than thirty frames per second (NTSC drop frame) or 
twenty-five frames per second; rates detemiined primarily by the frequency of 
AC mains supplies. It is also well known that high quality cinematographic 
film is displayed at twenty-four ft^mes per second, therefore there is an 
inherent difficulty in tenns of converting film Images Into video data. 

A process of 2:3 insertion for converting from twenty-four frames per 
second to thirty frames per second is well known and results In five output 
frames being generated from four input frames on a repeating c^de. Input 
frames making up the cycle are usually referred to as A, B, C and D frames 
from which five output finames are generated. A forther complication wth 
commercial video systems Is that each frame is generated fliom two 
interlaced fields therefore it is more correct to think in terms of generating ten 
output fields from four input frames. 

The known 3:2 insertion process generates three output frames that 
are derived fix)m a whole input frame. However, In addition to this, two flames 
are constructed in which intertaced fields are derived irorn different input 
frames such that when viewed as a still frame, there is a tendency for the 
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image to jitter, particulariy if movement has occurred between the two source 
frames. 

After 3:2 inserted video material has been produced, it may be 
necessary to perfbmn post production activities resulting in material being 
modified. In order to do this, the inserted video is reconverted back into 
material at twenty-four frames per second whereafter (theoretically) edits 
could occur at any frame traundary. However, given that material must be 
written back to an inserted video stream, taiown systems only allow edits at 
the boundaries between a D Frame and an A Frame, that is to say, at the 
start of the insertion cycle. Consequentiy, material that has been processed 
in this way introduces a further limitation into the editing process which may 
result in artistic compromises being made or may result in the Introduction of 
artefacts. 

Brief Summary of the Invention 

According to an aspect of the present invention there is provided 
apparatus for editing original video data of a first format, wherein said first 
fonnat has inserted jitter frames and is displayed at a first frame rate, 
comprising reading means configured to read original video data in said first 
fonnat from a video storage device; converting means for converting said 
video data read by said reading means into converted video data having a 
second fonnat with no jitter frames, displayed at a second display rate; 
relating means configured to relate time-code of said converted video data 
to time-code of said original data so as to identify frame types for said 
converted video data; modifying means configured to modify portion of said 
converted video data to produce modified video data in said second format; 
generating means configured to generate new video data in sakj first format 
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by processing said modified video data; and edit processing means 
configured to replace an edit portion of stored original data with said new 
data by defining an edit range with reference to said new data, selecting an 
edit field with reference to said frame types so as to initiate said edit on a 
data boundary, write said new data to said storage device, and select an 
edit field again for completing said process of writing said new data with 
reference to said frame type data so as to complete the edit on a data 
boundary. 

Brief Description of the Several Views of the Drawings 

Figure 1 shows a system for recording cinematographic film onto 
digital video tape; 

Figure 2 illustrates ttie fomnat conversion process identified in Figure 1\ 
Figure 3 shows a video editing facility; 

Figure 4 shows a computer controlled editing station of the type 
identified in Figure 3; 

Figure 5 illustrates operations perfonned by the system shown in 
Figures 3 and 4; 

Figure 6 illustrates an example of an editing operation; 

Figure / details an ovenA^riting operation identified in Figure 5; 

Figure 8 details procedures for determining a field edit type, identified 
in Figure 7; 

Figure 9 illustrates procedures for determining a field edit type at the 
end of an edit procedure; 

Figure 10 illustrates instmctions supplied over a serial internee to a 
video tape recorder. 



% 1^ 



4 

Best Mode for Carrying Out the Invention 

^ffSJstem tor recorcling cinematograpRrc nim'b'nioi^ 
illustrated in Figure f . Cinematographic f\\m^^;k<'4^iT^^guTe6 to be 
displayed at a rate of h^^^feor^flSn^per second is received by a film 
scanneMJP^'^eSiifi^ to generate high definition video signals at a off- 

Film scanner 102 includes a fomnat conversion process 103 
arranged to convert image signals scanned at 23.98 frames per second 
(notionally twenty-four frames per second) into a digital video signal with 
frames at 29.97 frames per second (notionally thirty frames per second). 
The thirty frame per second digital video signal is supplied to a digital video 
tape recorder 104 and both the film scanner 102 and the digital video tape 
recorder 104 are controlled by a control computer 105 over RS422 serial 
interfaces 106 and 107. 

Format conversion process 103 is illustrated in Figure 2. Film 
scanner 102 produces a progression of fi^mes derived from images 
captured on film 101. These are effectively a progression of still frames 
captured at a particular instant in time and therefore differ from a video 
signal generated by a video camera which produces a frame by scanning 
the image as two interlaced rasters such that, in effect, each pixel of data 
represents a portion of the image captured at a unique point in time. 
Progressive frames produced by the film scanner are initially generated at 
twenty-four frames per second and as such are usually identified as 24P 
frames, referenced 201 to 209. In order to perform a conversion process, a 
first frame 201 is identified as a type A frame, with the next frame 202 being 
identified as a type B frame, the next frame 203 being identified as a type C 
frame while frame 204 is identified as a type D frame. This sequence ABC 
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D then repeats such that frame 205 is identified as a type A, frame 206 is 
identified as a type B. frame 207 is identified as a type C. frame 208 is 
identified as a type D and frame 209 is identified as a type A etc. 

The progressive frames are generated at a rate of twenty-four 
frames per second ttierefore the cycle of four frames repeats six times each 
second. Digital video tape recorder 104 requires digital video signals at a 
rate of thirty frames per second therefore during the repetition period it is 
necessary to generate five video frames made up of a total of ten video 
fields. The fields are interlaced and thereby occupy odd or even lines 
respectively, the odd line field being referred to as a first field type and the 
even line field being referred to as a sea)nd field type. In this example, 
progressive frames 201 to 204 provide source material for generating 
output fields 211 to 220. When generating these fields, video data from the 
progressive frames is effectively inserted into field positions, with a first field 
of data being identified by a similar letter to its originating frame while a 
second field of data is identified by a similar letter followed by a star. Thus, 
both the fields 211 and 212 are derived firom frame 201 and the fields 
making up this frame are identified as A and A* respectively. 

At the video tape recorder 104, each of the generated frames is 
allocated a time-code, such as codes 223 to 227. In their full form, these 
time-codes represent hours: minutes: seconds: frames but in the example 
shown only entries for seconds and frames have been Included. In the 
majority of situations, time-codes would often indude an off-set such that, 
for example, the full time-code 223 may effectively start at the tenth hour 
and be fijily represented as 10:00:00:01. Notionally, tiie originating 
progressive frames 201 to 209 also indude time-codes, such as code 231 
and in the system described sut)sequentiy, these codes are effectively 
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reconstituted. 

As illustrated In Figure 2, a first output ftanne with time-code 00:01 is 
derived exclusively from input frame 201 or. looking at this the other way 
round, frame 201 inserts two data fields into the output stream. 

Input frame 202 is dealt with differently In that it inserts ttiree ou^ut 
fields into the data stream consisting of field 213 and field 214, to make up 
output fi^ame 00:02 while also making a contribution to output frame 00:03 
by the insertion of data to fiekJ 215, this being the same data as inserted in 
field 213, Thus, on this occasion there has been a three field insertion. 

Input frame 203 is processed differently again in that it provides a 
two field insertion but these are made to different output frames, consisting 
of a second field insertbn to position 216 and a first field insertion to 
position 217. The cycle is completed by a further three field insertion 
derived from input flrame 204 and consisting of the generation of output 
fields 218, 219 and 220. 

In this example, the high definition input film has been scanned to 
produce high-definition output video data at thirty per frames per second. In 
altemative configurations, standard video data may be produced and when 
produced in accordance with NTSC recommendations, this may also include 
drop frame time-codes, accomplished by making some output time-codes 
illegal as is well known in the art. It is also known to produce high definition 
video signals with drop frame time-codes. Furthemiore, having produced 
high<iefinition video signals this may also be further reconverted to standard 
definition broadcast video signals, possibly following different standard 
recommendations. Thus, a high-definition video source may be used to 
generate NTSC signals and PAL signals in addition to providing a source for 
high-definition broadcasts. 
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Techniques for the manipulation and editing of digitised video data are 
well known and such a system is licensed by the present Assignee under the 
Trademark "INFERNO", This facilitates a significant degree of post 
production activity allowing video to l>e edited and allowing modifications to 
be made, such as the inclusion of graphics and more complex special 
effects. Thus, many situations arise in which, after digitisation into a digital 
video signal, there is a requirement for further editing and modification to be 
performed prior to a final video output tape being produced. 

A video editing facility is installed in Figure 3. Digital video material is 
replayed from a standard digital video tape recorder 301, possibly similar to 
tape recorder 104 shown In Figure 1. Digital video images are supplied to a 
computer control and edit facility 302 over a digital video interface 303. The 
digital video tape recorder 301 is also controllal by the computer control and 
edit ^cility 302 via a serial RS422 control interface 304. In order for 
manipulations to be perfonmed. the computer control and edit facility 302 is 
interfaced to a display monitor 305 and manual input devices 306. 

A computer controlled editing station of the type illustrated in Figure 3 
is detailed in Figure 4, including a high definition video tape recorder 401. 
Video data at thirty frames per second, twenty-four frames per second or any 
other supported frame rate is written to and read from a plurality of randomly 
accessible discs 402 ananged in a redundant an^ay. The system is built 
around a computer system 403 such as an Onyx II computer manufactured 
by Silicon Graphics Inc. 

An operator controls applications perfonmed on the processing system 
403 by means of a stylus 404 applied to a touch tablet 405 in combination 
with the keyboard 406. Images and control menus are supplied to a high 
definition visual display unit 407 and video images are also supplied to a 
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broadcast quality monitor 408. Operating instructions accessible by the 
processing system 403 are received by means of a computer readable 
medium, such as CD-ROM 411 receivable within an appropriately configured 
CD-ROM player. 

5 Operations perfomied by the computer-based system illustrated in 

Figure 3 and detailed in Rgure 4 are identified in Figure 5. At step 501. an 
operator locates a frame reference with respect to a jitter frame. At step 502 
data is captured at thirty frames per second with time-code (or with drop 
frame time-code), effedlvely in a first fomnat At step 503, the system 
10 converts the video data capfcire at step 502 to progressive data at twenty- 
four frames per second again with time-code and this may be considered as 
being In a second format. 

At step 504 manipulations are performed on the reconverted data 
which may take the fonii of an edit or a re-composition. Edits would not 
15 generally increase or reduce the lengOi of a particular dip but they would 
result In material being modified and thereby require this modified material to 
be written back to the digital tape recorder 301 at appropriate locations. 

At step 505 the modified second fonnat dip (I.e. 24P) Is identified as 
the portion of the dip that is to be written back to the tape recorder. At step 
20 506 this clip Is converted again into the first format i.e. the thirty fi^me per 
second jitter frame inserted format 

Steps 506 and 507 effectively take place in unison, with the system 
perfonming the 2:3 insertion process as the material is being written onto the 
tape, based on the reference A frame time-codes. 

fl=!e'-video'''@pg-gd!t''g'"^f^^ bU/ by tne nmf ^k.^^^ 

overwriting an appropriate portion of^^g^riginal data such that the 
modification is then induded oi:i«>lhe material held on the digital video tape, 
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Performing modifications upon the 24P data, converting ttiis data to 
thirty frames per second inserted and then performing an edit to video tape is 
relatively straightforward if edits only occur at the boundaries between D and 
A frames, such as boundaries 251, 252 and 253 as sho\Am in Figum 2, 
However, if edits are required at other frame boundaries, problems arise 
which, in previous systems generally resulted in the introduction of artefacts. 
A problem occurs, for example, if an edit is to start at a boundary between an 
A frame and a B frame, say, and then finish at a boundary between a C 
frame and a D frame. 

It Is possible for a video tape recorder to be instructed such tiiat edits 
occur on the first field of a frame or on the second field of a frame. When 
edits occur on the first field, the cuts effectively take place along frame 
boundaries such that, for example, the frame made up of fields 211 and 212 
of Figum 2 would remain intact and edits could only occur on either side of 
these fields. Similariy, when edits occur on the second field, it would not be 
possible to put a break between fields 212 and 213, actually in different 
frames, although an edit could occur between fields 211 and 212 or between 
fields 213 and 214, effectively breaking the notional frame in half. 

When edits are specified by time-code, the code at the start of the edit 
is indusrve and that at the end of the edit is exdusive, therefore a set-in time- 
code represents a time-code of the first frame to be replaced and a set-out 
time-code represents the time-code of the frame after the last firame to be 
replaced. In the editing system, edits are defined at boundaries of 24P 
frames, after which the system is configured to reconstitute the edit from 2:3 
pull down inserted frames in such a way as to foithfully restore the edit in the 
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first format stored on video tape. 

An example of an edit is illustrated in Figure 6. A clip consists of a 
sequence of 24P frames 601, 602. 603, 604 etc to frame 605 having time- 
codes 03:12 to 05:09; hours and minutes not being shown in order to 
improve clarity. Following operations perfbnned to find a reference "A" frame 
on tape, the frame types are known, with frame 601 being of type D. frame 
602 being of type A, frame 603 being of type B, frame 604 being of type C 
etc, with frame 606 being of type C, frame 607 being of type D and frame 605 
being of type A. 

Manipulations have been performed on the computer system and from 
this it has been determined that the new material should be set-in from frame 
03:14 and set-out at frame 05:07. In order to achieve an edit of this type 
without introducing arte^cts, all material derived from frames 603, 604 and 
all the frames in-between up until frame 606 is to be replaced, whereas 
material derived from frame 602 and earlier frames along with material from 
frame 607 and later frames is to be retained without modification. 

24P frames 601 to 605 were generated from inserted frames 621 to 
629 but frames 603 to 606 have been modified therefore similar modifications 
need to be made to appropriate frames/fields of the inserted video so that the 
appropriate portion of modified inserted video may be then written back to the 
tape recorder. Frame 622 at time-code 04:16 is derived from frame 602 at 
time-code 03:13 similariy, frame 623 is derived exclusively from frame 603. 
This facilitates an edit of the video material given that an edit cut may occur 
at the boundary between frame 622 and frame 623. 

On the set-out side frame 606 has been modified and frame 607 has 
not been nrKxJified. Thus, material derived from frame 607 is to be retained 
and not ovenvritten whereas material derived from frame 606 has been 
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modified and is therefore to be overwritten on the tape. On this occasion 
however the cut does not occur at a frame boundary given that frame 627 is 
a jitter frame, having received an insertion from frame 606 with a 
complementary insertion from frame 607. A problem therefore exists In that 
the set-in edit, between frame 622 and 623 occurs on the first field with the 
set-out edit at frame 627 occurring on the second field. 

The invention overcomes this problem of perfomiing an edit upon 
video data of a first fomnat In whidi the first format has Inserted jitter frames 
and is displayable at a first fi^me rate. The original video data in the first 
fomiat is read from a storage device, such as a video tape recorder. The 
video data is converted into converted video data having a second fomiat 
with no jitter frames thus, having fewer fliames removed, the second fomnat is 
displayable at a second display rate. Preferably, the Image data is written to 
randomly accessible storage and a time-code is allocated to l>oth the first 
fomiat frames and the second fomnat flrames. Having generated the second 
fomiat data, this data is then available for modifications to be made such that 
a portion of the converted video data may be nrKKllfied to produce modified 
video data in the second fomiat 

However, this modified data cannot be written back to the video tape 
in its converted form. The generation of new data, i.e, back Into its first 
fomnat, is perfomned while data is being written back to the vkJeo tape 
recorder and the processing is effected in real-time. An edit portion on video 
tape is then replaced by the new data by performing an editing process. 
During this editing process an edit range is defined with reference to the new 
data. The tape recorder is responsive to field edit commands and as such an 
edit field is selected with reference to the frame type so as to initiate the edit 
on a data boundary. Witii this data boundary selected, new data is written to 
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the storage device ensuring that the cut point In the first fomiat video is 
conrect with reference to the specified cut point in the second fomiat video. 
As shown in Figure 6, this does not in itself provide a total solution, given that 
the edit field at the end of the edit may be different from the edit field at the 
b^inning of the edit. Consequently, while data is being written to the video 
tape, the edit field is selected again before completing the writing step, with 
reference to the frame type data so as to complete the edit on the data 
boundary. 

Thus, by maintaining a mapping of time-codes related to an A-ftBme 
reference and thereafter calculating frame types, it is possible to detemiine 
what type of field edit is require at the set-in point and at the set-out point. 
Furthennore, if ttie field type between these two points differs, the field edit 
type is changed while the data is being transferred so as to ensure that the 
conrect type of field edit occurs at the set-out point 

Step 507 for ovenvriting original data on video tape with the new data 
generated by the computer system is detailed in Figure 7. 

lAferi stopiaCMdtbfe Bew^^ 
video) is considered to identify a time-code for the start of the e^ftf^t step 
702, this time-code is compared witii time-codes timed^?^ to the "A" 
frame reference time-code to detemiine a startjEa^ type. Thus, refening to 
Figum 6, the time-code of frame 623^Jj^^ified that in turn corresponds to a 
start frame type of BB* thereb^'^iraking frame 603 a "B" frame. This is 
identifies the secondfo^^i^f^rt frame type at step 702. Thus, at step 703, a 
determination is?:?fnade as to what type of field ^it is required. In this 
exampl%:^en that tiie edit starts on a B type frame, a first field ^it type is 
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At step 704 a similar process is initiated for the end of the edit and a 
time-code Is identified for the last first forniat type fiame Included in the edit. 
In the example shown in Figure 6, this occurs at frame 627 which is derived 
firom frames 606 and 607. Thus, on this occasion, a time-code is identified as 
06:06 at step 704. 

At step 705 the frame type is identified as a C type, i.e. frame 606 
whereafter at step 706 the field edit type is identified as requiring an edit on 
the second field. This is to ensure that the material from frame 606 is 
replaced while the material from frame 607 is retained. It should be noted that 
first format set-in time-codes on DD* frames are not possible since the D 
frame does not begin there. Also, set-out Ume-codes on BC* frames are not 
possible since no second fomnat frames end on the preceding BB* frame. 

At step 707 the video tape recorder is Instructed to initiate the edit and 
at step 708 video data is transfemed. At step 709 a question is asked as to 
whether the edit field type is con^, that is to say. whether the field type 
determined at step 706, for the end of ttie edit, is the same as the field type 
detennined at step 703, for the beginning of the edit If these field types differ, 
the question asked at step 709 is answered in the negative and the field type 
edit Is changed at step 710. Thereafter at step 711 a further instruction is 
supplied to the tape recorder changing the field type edit so as to ensure that 
a conrect type of edit occurs at the set-out point Alternatively, if the question 
asked at step 709 is answered in the affinnative, no further action is required 
and control continues to step 508. 

Step 703 for detemiining the field edit type at the start of the edit 
process, i.e. the setwn point, Is detaOed in Figure 8. 

At step 801 a question is asked as to whether the edit starts on an A 
frame and if answered in the affirmative, a flag is set equal to one, to confirm 
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that the field edit is to take place on the first field. Alternatively, rf the question 
asked at step 801 is answered in the negative, control is directed to step 803 
at which a question is asked as to whether the edit starts on a B frame. 
Again, rf answered in the affirmative, the flag is set equal to one, representing 
a condition to the effect that the edit is to take place on the first field. 

Altemativety, If the question asked at step 803 is answered in the 
negative, control is directed to step 805 and under these conditions the edit 
will be initiated on a C frame or a D fi^me requiring the edit to be effected on 
the second frame resulting in the flag being set equal to zero. Thus, if the edit 
does not start on an A frame or does not start on a B frame the edit must be 
on a C or a D fi^ame requiring the cut to take place on the second field. 

Step 706, for determining the field edit type at the end of the edit, is 
detailed in Figum 9. At step 901 a question is asked as to whether the edit 
ends at the end of an A frame. If answered in the affinmative, control is 
directed to step 902 resulting in the field flag being set equal to one 
representing a condition to the effect that the edit takes place on the first 
field. 

If the questton asked at step 901 is answered in the negative, control 
is directed to step 903 where a question is asked as to whettier the edit ends 
at the end of a D fi-ame. Again, if answered in the affimiative, the fieW flag is 
set equal to one representing a condition to the effect ttiat a first field edit is 
required. If the question a^ed at step 903 is answered in the negative, the 
edit must end on a B type frame or a C type frame under which conditk)ns an 
edit Is required on the second field resulting in ^e field flag being set equal to 
zero at step 905. 

Step 707 of Figure 7 generates instructions to the video tape recorder, 
over the serial interface 304. of the type illustrated in Figure 10. At line 1001 
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a set-in instruction Is sent instnjcting the video recorder to initiate an edit from 
the frame defined by the time-code calculated at step 701 . At line 1002 a set- 
out Instruction is Issued to the video tape recorder identifying a time-code for 
the end of the edit derived flnom the calculation perfomied at step 704. 
5 At line 1003 edit tracks are identified usually in the fomn of a video 

track plus several audk) tracks. The particular tracks under consideration are 
defined during the manipulation of the vkleo data at step 504. 

At line 1004 an instoiction to "set the edit field" Is issued derived from 
the detenmlnation made at step 703. Thus, line 1004 identifies whether the 

10 edit Is to be perfomied on the first field (F1 ) or on the second field (F2). 

At line 1005 the physical editing process is initiated by a command to 
cue to In pre-rolL This rewinds ttie tape recorder in accordance with 
established pre-roll techniques ensuring that the mechanism of the machine 
Is fully in synchronism with the input video stream |:^or to the actual edit 

15 being initiated. This is then followed at line 1006 by the command to "auto 
edir. 

As prevtously describe, if the questbn asked at step 709 is answered 
In tfie negative, further actfon must be taken in order to ensure that the edit 
correctly terminates at the required field position. At step 710 the field edit 

20 flag is changed resulting in a further instruction being issued at step 711 as 
shown in Hgure 10. This takes the fbmi of a further command at line 1011 
similar to the command issued at line 1004, taking the form *'set edit field" but 
on this occasion effectively deriving the field edit data firom step 706. 

Thus, these procedures facilitate accurate insertions to be made 

25 during an editing process such that editors are not restricted when source 
material includes inserted jitter frames and field accurate edits can be 
accomplished without the introduction of any artefacts. 



